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Characteristics of School Safety and ACT 
Test Performance 


Jeff Schiel, PhD 


Unsafe schools can negatively impact students’ wellbeing and educational success. In 
an introduction to their edited book on school safety and violence prevention, Mayer 
and Jimerson’ describe research indicating that there may be multiple short- and long- 
term detrimental effects on students who experience school violence, disruption, and 
bullying. These include such things as increased anxiety, cognitive processing difficulty, 
reduced motivation and attention, health issues, learning problems, and lower academic 
performance as measured by grade point average, standardized test scores, and 
graduation rates. 


This study focuses on one of the potential effects of unsafe schools, decreased 
standardized test scores. Recent evidence that standardized test scores can be 
negatively affected in unsafe schools was provided in a large-scale analysis of student 
survey and education record data from more than 700 New York City middle schools.” 
That analysis found that students who reported feeling unsafe in the classroom earned 
lower standardized test scores than did students who reported feeling safe. Moreover, 
students who reported that they stayed home due to feeling unsafe earned even lower 
test scores, indicating the presence of both direct (i.e., feeling unsafe) and indirect (i.e., 
increased absences) negative effects on academic achievement. 


Safe schools, in comparison, appear to make a difference even in unsafe 
neighborhoods. Research suggests that school climates that are positive and provide 
students with a place where they can feel insulated from neighborhood violence are 
associated with positive learning outcomes.? It was found that students who attended 
such schools earned higher English language arts standardized test scores than did 
students who attended schools viewed as less safe or as having a relatively weak sense 
of community. 


It is likely that performance on the ACT® test is related, to some extent, to students’ 
perceptions of safety at school. This study sought to quantify that relationship, with 

the goal of providing insights for schools, by surveying a large sample of high school 
students who took the ACT and asking them questions about the safety of their schools. 
A total of 14,903 of these students responded to the survey.* 
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Key Findings 


After statistically controlling for student background characteristics via multiple linear 
regression, it was found that several school safety characteristics were related to 
performance on the ACT test. Equally noteworthy is that a larger number of school 
safety characteristics were found not to be related to ACT performance. 


In particular, it was found that not requiring metal detector checks before entering the 
school building each day, having school security staff, explaining school building-wide 
emergency plans to students, having a perceived low presence of gangs at school, and 
having low concerns about safety at school negatively impacting students’ ability to 
learn were all positively related to performance on the ACT (See Figure 1; additional 
information about the first three of these characteristics is provided in Figure 2). 
Characteristics that were unrelated to ACT test performance include, for example, the 
extent to which students feel welcome and safe at school; teachers’ ability to manage 
out-of-control students; and whether schools are locked during school hours, have 
security cameras, and provide mental health services for students.°® 


Figure 1. Weighted, Unstandardized Regression Coefficients for Modeling ACT Composite 
Score as a Function of School Safety and Student Background Characteristics (Final Model) 


My school requires metal detector checks [I— —4 
My school has security staff 
My school has explained its emergency plan 


There are gangs at my schoo!l* 


TT ey 


Concerns about my safety at school neg. impact ability to learn* 
Whether or not a fee waiver was used for the ACT Fe 
Rural (1) vs. urban (0) school location Fe4 
Suburban (1) vs. urban (0) school location Fed 
Town (1) vs. urban (0) school location Fe 
Minority (1) vs. White (0) Fe 
Asian (1) vs. White (0) -—e-—_| 
Two or more races (1) vs. White (0) HKe— — 
All other races (1) vs. White (0) - -—e—__¥+ 
Female (1) vs. male (0) Fe4 
Expected educational attainment | 
High school GPA inal 
Family income | 
Having taken math courses beyond algebra II IK— —4 
Having taken biology, chemistry, and physics be 
Highest level of parental education | 
School affluence indicator lel 


-3 -2 -1 0 1 2 3 


Regression coefficient (95% confidence interval) 


Note. The dashed line in Figure 1 separates the school safety characteristic regression coefficients 

from the student background characteristic regression coefficients. The regression coefficients are not 
intended to indicate the relative importance of any of the characteristics, but they do indicate the direction 
of the relationship with ACT Composite score. For example, requiring school metal detector checks 

has a negative regression coefficient (-1.24) and was therefore associated with an average decrease 

of 1.24 ACT Composite score points, while statistically controlling for other student and school safety 
characteristics. 


*These survey items were reverse-coded for analysis purposes. 
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Note the positive regression coefficient for the gang activity characteristic. This particular 
survey item was reverse coded for analysis purposes, so that higher values were 
assigned when students disagreed with the statement “there are gangs at my school,” 
and lower values were assigned when they agreed. The positive regression coefficient 
therefore indicates that less gang activity (i.e., students generally disagreeing with the 
statement) was associated with an average increase in ACT Composite score. The 

item about safety concerns negatively impacting ability to learn was also reverse-coded. 
Table 1 in the Technical Appendix provides additional information about the coding of the 
survey items. 


The first three safety characteristics depicted in Figure 1 (metal detector checks, security 
staff, and an emergency plan that is explained to students) differ from the others in that 
students were asked to indicate whether these characteristics were present or not at 
their schools, rather than to agree or disagree with statements about the characteristics. 
This allows for an estimation of the effect on ACT test performance when the 
characteristics are present, compared to the effect when they are not. Further analysis 
revealed that students attending high schools that have a building-wide emergency 
plan that was explained to the students, school security staff, and do not require 

metal detector checks before entering the building each day were likely to experience 
somewhat higher average performance on the ACT, compared to students attending 
high schools without these characteristics (Figure 2). 


Figure 2. School Safety Characteristics and Average ACT Test Performance 


My school ... 


Has explained the building-wide emergency plan 
Has not explained the building-wide emergency plan 


Has security staff 
Does not have security staff 


Requires metal detector checks before entry 


Does not require metal detector checks before entry 


1 6 11 16 21 26 31 36 


Adjusted Average ACT Composite Score 


These findings apply irrespective of students’ race/ethnicity, gender, family income, 
expected educational attainment, and high school grade point average; whether a fee 
waiver was used for the ACT test or not; geographical area in which students’ schools 
are located (e.g., rural, urban); the level of affluence of their schools®; whether math 
courses beyond algebra II were taken or not; whether biology, chemistry, and physics 
were taken or not; highest level of education achieved by a parent/guardian; and 
additional characteristics, such as students’ perspectives on the presence of gangs 
at their schools and whether or not concerns about their safety at school negatively 
impacted their ability to learn.’ All these student and school characteristics were 
statistically controlled in this study. 
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Discussion 


Building on prior work in this area, this study found that students’ perceptions of school 
safety for several of the characteristics examined were positively related to academic 
achievement, as measured by standardized test scores. However, the results were 
nuanced in that some characteristics of school safety showed no substantive relationship 
with ACT performance. For schools that are considering implementing and/or changing 
safety characteristics, this finding is especially important to keep in mind. In addition, 
even though a particular safety characteristic may have been found in this study to be 
related to ACT performance, consideration of several factors should precede a school’s 
implementation or changing of that characteristic. 


For example, should schools that do not have an emergency plan that has been 
explained to students, do not have school security staff, or do require metal detector 
checks before entering the building each day implement and/or change one or more 
of these characteristics to try to improve students’ performance on the ACT? Not 
necessarily. 


This study focused on the relationship of school safety characteristics to ACT Composite 
score. However, there are almost certainly other aspects of the student experience, 
besides ACT test performance, that should be considered before implementing or 
changing any safety characteristics. Moreover, students will likely have differing opinions 
of the usefulness of such safety measures as school security staff and metal detector 
checks. Students’ widely varying perspectives on school safety measures prompted 
Croft, Moore, and Guffy? to recommend that policymakers ensure the collection and use 
of student input when deliberating the implementation of such measures. 


The implementation of school security staff may require particularly careful forethought, 
given recent research indicating that increasing school resource officers is associated 
with immediate increases in weapon- and drug-related offenses, which persist for 20 
months after the resource officer increase occurs.® Moreover, the quality of security staff 
may vary, at least from the perspective of students. When examining student responses 
to an open-ended question about the safety of their schools, Moore, Croft, and Heisdorf'® 
observed that some students were concerned about security staff members’ lack of 
preparation for an emergency, lack of professional security training, and distractions from 
focusing on student safety as a result of other responsibilities, such as teaching courses. 
Other students, in comparison, described positive experiences with security staff quality, 
including their ability to make students feel safe and welcome. It is important to keep in 
mind that the survey instrument in the present study, which was also used in the Moore 
et al. (2020) study, asked not about school resource officers but about “security staff,” 
which could be broadly defined from the perspective of students and may or may not 
include school resource officers. 


Although the observed differences in average ACT Composite score in this study 
indicate a relationship with school safety characteristics, they do not provide evidence 
of causality. For example, although larger average adjusted ACT Composite scores 
occurred for students in schools that have safety staff, compared with students in those 
that do not, the presence of school safety staff was not demonstrated in this study to 
cause the larger scores. 
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It is interesting that requiring daily metal detector checks prior to entering the school 
building was associated with a decrease in ACT Composite score, on average. This 
finding could be related to the decreased perception of school safety associated 
with the presence of metal detectors observed in some studies (for example, see: 
Perumean-Chaney and Sutton"; and Hankin, Hertz, and Simon’”). 


It is possible that unobserved factors might be related to some of this study’s observed 
effects. For example, school factors that were not addressed in the survey and 
therefore were not included in the analyses could be related to the observed negative 
association between daily metal detector checks and ACT performance. 


Before deciding to either change existing school safety characteristics or implement 
new ones, schools are strongly encouraged to collect their own data and to draw 

their own conclusions about relationships between school safety characteristics and 
performance on the ACT. Although some school-level data were analyzed in this study 
and used for statistical control purposes, there are likely other, local data that schools 
may wish to incorporate in their own analyses. The use of such local data could result 
in findings that differ from those observed in this study. 


Technical Appendix 


The Survey and the Student Sample 


In October 2018, ACT researchers surveyed a sample of approximately 95,000 
students in grades 10-12 who had registered to take the National ACT test. The 
purpose of the survey was to gauge students’ perception of the physical safety of their 
schools. The sample was stratified on gender, race/ethnicity, and school geographical 
area (urban, suburban, rural, town), resulting in 50 strata. Simple random sampling 
was performed within each stratum. In general, the number of students sampled from 
a particular stratum was disproportional to the number in the target population for that 
stratum. This was done intentionally to ensure that the number of respondents in each 
stratum was sufficient for subgroup analyses. For example, researchers might want 

to compare the survey responses of female, Hispanic students from suburban areas 
with those of female, Hispanic students from urban areas. Croft et al. (2019) provide 

a report of students’ perceptions of physical safety at school based on the data from 
this survey, with some findings disaggregated by the demographic variables mentioned 
above. 


Approximately 16,000 students completed the school physical safety survey. Some of 
these students did not actually take the ACT test, even though they were registered to 
do so. A total of 14,903 students completed the survey and took the test. 


Data 


The survey responses of each student were combined with their ACT test record 

and registration data, which include test scores and self-reported background 
characteristics. In addition, school-level data (e.g., school geographic area and 
enrollment counts) from MDR’s education database were appended to student records. 
(MDR is a division of Dun & Bradstreet specializing in education data.) Standard 
statistical methods for imputing missing values were used to ensure that data from all 
14,903 students were represented in the analysis. 
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Multiple Linear Regression 

Multiple linear regression was used to model ACT Composite score as a function 

of student background characteristics and survey response data (Table 1). The 
regression analyses incorporated features of the complex sampling design described 
above, including sampling weights that adjusted for disproportional sampling within 
strata. 

Table 1. Student Background Characteristics and Items from Physical Safety Survey Instrument 


Student Background Characteristics 


(olalanteTalhVarsx>)1eg k=) ele) atate pmexe)|(=Yeu-to mui al=1a 
students register for the ACT test) Items from Physical Safety Survey Instrumentt 


*Race/ethnicity* . | feel welcomed at my school 


1 
°Gender* 2. Theft doesn’t happen much at my school 
spade level 3. My school has rules that keep people safe 
4. | feel safe at school 

5 


SExPEcied calicalignal Aucinnens . Teachers are able to manage students who get out of 


*Having taken math courses beyond control 
algebra II 6. People rarely get physically hurt at my school 
*Having taken biology, chemistry, and 7. None of the students at my school carry weapons 
physics* 8. Students at my school don’t get in trouble with the law 
¢Highest level of parental education* 9. Students treat the teachers with respect at my school 


+Family income* 10. There are gangs at my school* 


11. School classrooms and hallways are clean 
*High school GPA* 


12. | feel safe traveling between home and my school 

pi can area of school (e.g., urban, | 43. Students from various racial/ethnic backgrounds get along 
rural) well with each other at my school 

*Whether or not a fee waiver was used for | 14. Students of different sexual orientations get along well with 
the ACT* each other at my school 


*School affluence indicator’”* 15. Concerns about my safety at school negatively impact my 
ability to learn* 


. My school requires metal detector checks before entering 


*School poverty level” the building each day* 
17. My school is locked during school hours 


18. My school has security cameras inside the school 


«School enrollment’ 16 


¢School type (public, private)* 
; bases Sen 
Soleel pel pupibexpengiate 19. My school has security cameras outside the school 


*ACT Composite score (dependent 20. My school has security staff* 
variable)* , 
21. My school only allows students to carry clear/see-through 


book bags 


22. My school has explained to me our building-wide 
emergency plan* 


23. My school provides mental health services for students who 
need them 


tScale for Items 1-15: 6=strongly agree, 5=agree, 4=somewhat agree, 3=somewhat disagree, 2=disagree, 
1=strongly disagree (scale is reversed numerically for items 10 and 15). For Items 16-23, students 
indicated which of the school safety elements pertained to them. 


*Included in final regression model 

Obtained from MDR 

Obtained from MDR; this variable is based on a proprietary algorithm incorporating various data points, 
including census data, to rank the socioeconomic status of a school. 
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A total of 23 items pertaining to students’ perception of physical safety in school were 
examined in regression models. Using standard methods for determining whether to include 
an independent variable in a regression model (e.g., statistical significance, 

degree of multicollinearity), 5 of the 23 items were identified for inclusion in the final 

model. 


Descriptive statistics for the student background characteristics and physical safety 
items included in the final regression model are provided in Table 2, and corresponding 
regression statistics are provided in Table 3. 


Table 2. Descriptive Statistics for Student Background Characteristics and School Physical Safety Items 


NAYet(e [al (Xe) 
Percent (or 

Characteristic/Item eee Tad) Weighted N 
Race/Ethnicity 

American Indian/Alaska Native 0.9% 130 

Asian 5.7% 847 

Black/African American 14.8% 2,204 

Hispanic/Latino 18.7% 2,786 

Native Hawaiian/Other Pacific Islander 0.1% 19 

White 51.3% 7,645 

Two or more races 5.2% 780 

Prefer not to respond 3.3% 493 
Gender 

Female 59.4% 8,857 

Male 40.6% 6,046 
Expected Educational Attainment 

Business/technical school or certificate program 0.7% 111 

Associate degree 2.2% 321 

Bachelor's degree 47.9% 7,131 

One or 2 years of graduate study (MA, MBA, etc.) 19.4% 2,888 

Doctorate or professional degree (PhD, MD, JD, etc.) 29.9% 4,452 
Having Taken Math Courses Beyond Algebra II 

No 1.5% 230 

Yes 98.5% 14,673 
Having Taken Biology, Chemistry, and Physics 

No 61.8% 9,212 


Yes 38.2% 5,691 
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Table 2. Descriptive Statistics for Student Background Characteristics and School Physical Safety ltems—continued 


Weighted 
Percent (or 
Characteristic/Item NV CeeTaD) Weighted N 
Highest Level of Parental Education 
Less than high school 6.1% 913 
High school graduate/GED 13.5% 2,010 
Business/technical school or certificate program 3.0% 451 
Some college, no degree or certificate 11.0% 1,636 
Associate degree 9.5% 1,412 
Bachelor's degree 28.7% 4,283 
One or 2 years of graduate study (MA, MBA, etc.) 18.6% 2,771 
Doctorate or professional degree (PhD, MD, JD, etc.) 9.6% 1,428 
Family Income 
Less than $24,000 14.3% 2,136 
About $24,000 to $36,000 12.2% 1,818 
About $36,000 to $50,000 11.0% 1,638 
About $50,000 to $60,000 7.6% 1,136 
About $60,000 to $80,000 11.3% 1,690 
About $80,000 to $100,000 10.6% 1,579 
About $100,000 to $120,000 10.2% aor 
About $120,000 to $150,000 8.0% 1,200 
More than $150,000 14.6% 2,180 
High School GPA 
(D- to D) 0.5—0.9 or lower 0.0% ri 
(D to C-) 1.0-1.4 0.1% 19 
(C- to C) 1.5-1.9 0.6% 91 
(C to B-) 2.0—2.4 2.5% 32 
(B- to B) 2.5-2.9 7.9% 1,183 
(B to B+) 3.0-3.4 22.0% 3,210 
(A- to A) 3.5—4.0 or higher 66.8% 9,954 
Geographical Area of School 
Rural 15.2% 2271 
Suburban 36.7% 5,465 
Town 19.5% 2,903 
Urban 28.6% 4,265 
School Affluence Indicator 
Low 19.6% 2,918 
Below Average 19.6% 2,927 
Average 20.7% 3,088 
Above Average 21.1% 3,136 


High 19.0% 2,834 
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Table 2. Descriptive Statistics for Student Background Characteristics and School Physical Safety ltems—continued 


Weighted 
Percent (or 

Characteristic/Item NV eeTaD) Weighted N 
Fee Waiver Used for the ACT 

No 69.2% 10,310 

Yes 30.8% 4,593 
There Are Gangs at My School 

Strongly Disagree 35.6% 5,306 

Disagree 31.0% 4,620 

Somewhat Disagree 12.3% 1,800 

Somewhat Agree 11.7% 1,736 

Agree 6.1% 907 

Strongly Agree 3.076 498 
Concerns About My Safety at School Negatively Impact My Ability to Learn 

Strongly Disagree 28.5% 4,240 

Disagree 35.8% 5,a39 

Somewhat Disagree 13.0% 1,943 

Somewhat Agree 11.8% 1751 

Agree 7.3% 1,084 

Strongly Agree 3.7% 546 
My School Requires Metal Detector Checks Before Entering the Building Each Day 

No 96.7% 14,412 

Yes 3.3% 491 
My School Has Security Staff 

No 32.1% 4,788 

Yes 67.9% 10,115 
My School Has Explained to Me Our Building-Wide Emergency Plan 

No 34.6% 5,154 

Yes 65.4% 9,748 
ACT Composite Score (Mean) 22.8 14,903 


Note. Each percentage in this table is a weighted percentage, averaged over all 20 imputations. 
Similarly, the n-counts corresponding to each of the percentages are weighted n-counts 
averaged over all imputations. Due to rounding, the weighted percentages for some background 
characteristics might not sum to 100%, and the weighted n-counts might not sum to 14,903. 
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Table 3. Weighted Regression Statistics for Modeling ACT Composite Score as a Function of Student 
Background Characteristics and Survey Response Data (Final Model) 


Uyatcitclarer=|nelr4cve| 


Regression 
fatel=ver=yale(>1alMvcslat=le) (>) Coefficent cS) (0 fal =i 0) 
Intercept -0.89 0.55 -1.62 
Fee Waiver Used for the ACT -0.67 0.14 -4.91 
Rural (1) vs. Urban (0) School Location -0.86 0:12 -7.10 
Suburban (1) vs. Urban (0) School Location 0.09 0.12 0.77* 
Town (1) vs. Urban (0) School Location -0.62 0.14 -4.31 
Minority (1) vs. White (0) -1.87 0.11 -16.98 
Asian (1) vs. White (0) 1.06 OR 3.99 
Two or More Races (1) vs. White (0) -0.13 0.34 -0.38* 
All Other Races (1) vs. White (0) -0.74 0.42 -1.74* 
School Requires Metal Detector Checks (1=selected, -1.24 O.27 -4.66 
0=not selected) 
School Has Security Staff (1=selected, O=not selected) 0.63 0.10 6.01 
School Has Explained its Emergency Plan (1=selected, 0.31 0.10 3.14 
0=not selected) 
There are Gangs at my School 0.19 0.03 5.45 
Concerns About my Safety at School Negatively Impact 0.34 0.03 10.01 
my Ability to Learn 
Female (1) vs. Male (0) -1.01 0.10 -10.25 
Expected Educational Attainment 0.77 0.06 13.37 
High School GPA 2.07 0.06 31.98 
Family Income 0.28 0.03 10.14 
Having Taken Math Courses Beyond Algebra II (1=yes, 2.00 O22 629 
0=no) 
Having Taken Biology, Chemistry, and Physics (1=yes, 1:15 0.10 11.74 
0=no) 
Highest Level of Parental Education 0:31 0.03 ez 
Affluence Indicator 0.18 0.04 4.78 


*Although these variables were not statistically significant (p-values of 0.71 for two or more 
races vs. White, 0.08 for all other races vs. White, and 0.44 for suburban vs. urban), they were 
part of two series representing racial/ethnic group membership and geographical area of the 
school and were included for control purposes. 
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